The Use of Crow-AMSAA Plots to Assess Mishap Trends

Crow-AMSAA (CA) plots are used to model reliability growth. Use of CA plots has expanded into other areas, such as tracking events of interest to management, maintenance problems, and safety mishaps. Safety mishaps can often be successfully modeled using a Poisson probability distribution. CA plots show a Poisson process in log-log space. If the safety mishaps are a stable homogenous Poisson process, a linear fit to the points in a CA plot will have a slope of one. Slopes of greater than one indicate a nonhomogenous Poisson process, with increasing occurrence. Slopes of less than one indicate a nonhomogenous Poisson process, with decreasing occurrence. Changes in slope, known as "cusps," indicate a change in process, which could be an improvement or a degradation. After presenting the CA conceptual framework, examples are given of trending slips, trips and falls, and ergonomic incidents at NASA (from Agency-level data). Crow-AMSAA plotting is a robust tool for trending safety mishaps that can provide insight into safety performance over time. Introduction Crow-AMSAA (CA) plots can be used to track safety mishaps. Using this tool, one can trend mishaps over time and see if the mishap rate is stable or changing for better or worse. Predictions can also be made of the number of future mishaps over a given time period. First, a brief discussion of the historical development and theoretical aspects of CA plots will be given, followed by a few examples. The fitting of a Poisson probability distribution to a safety mishap dataset will also be shown, as well as predictions based on the fitted distribution. Historical and Theoretical Perspective CA plots are based on a power law function, which is derived in part from learning curves. The Duane model of reliability growth was an extension of the learning curve power law concept (ref. 1). The CA model, which is similar to the Duane model, started with development of reliability growth models by the Army Materiel Systems Analysis Activity (AMSAA). Dr. Larry Crow, in his paper entitled Reliability Analysis for Complex, Repairable Systems (ref. 2), considered the use of a nonhomogenous Poisson process for reliability analysis. A Poisson process can be used to model a more complex process, provided that certain assumptions are met. Assumptions of the Poisson model are (refs. 3-5): 1. The number of events in any time interval is independent of the number of events in another interval. 2. Events cannot happen simultaneously. This condition can be satisfied by considering a sufficiently small interval. 3. The probability of an event within a given time interval is proportional to the length of time of the interval. Repairable system reliability (and similarly, a safety mishap process) is the result of the interplay of complex factors and conditions. One approach to modeling a complex system is to use a simplified model. Homogenous and nonhomogenous Poisson process models allow us to simplify analysis provided that the model can be shown to adequately fit the data. The CA model, while initially derived for reliability growth, has been successfully applied to trending in other domains and is robust for use with deficient data. Use of a reliability growth model for trending safety mishaps is possible because both can be represented by a Poisson process. Safety performance, as measured by mishap counts, is an analog of reliability performance, as measured by failures. An increase in the rate of safety mishap occurrence is analogous to degrading reliability; a decrease in the mishap rate is analogous to reliability improvement. The Poisson distribution is useful in a variety of applications, including inventory control and queuing models. It was shown to be a good predictor of the number of people kicked to death by horses in the Prussian army (ref. 6). It was also shown to accurately model the number of bomb hits per area in London during the second World War (ref. 7). Often safety mishaps will follow a Poisson distribution closely enough for it to be a good choice for modeling them. For CA plots to be valid, we need to validate the assumption that the data are accurately represented by a Poisson process (i.e., the number of events follows a Poisson distribution). While an exact fit to a Poisson probability distribution is not required (or expected), a check for goodness of fit is recommended. Some data that might be expected to be adequately modeled by the Poisson may not be. The chi-square goodness of fit test is often used. The Kolmogorov-Smirnov test is an alternative. Application of Crow-AMSAA Plots to Trend Safety Mishaps Given a set of mishap data over time, we plot the cumulative number of mishaps versus time in a log-log plot. (Power law functions typically plot as a straight line in log-log plots.) For analytical convenience, the number of mishaps is accumulated by month (the chosen unit of time on the horizontal axis). The cumulative number of mishaps is the vertical axis. A straight line is then fitted to the plot. The best method of fitting the line depends on characteristics of the data set (ref. 8). We should take care that the data have been cleaned to eliminate data points that are not of the category of interest. Once a line is fitted to the CA plot, the slope of the line (beta) is interpreted as follows: beta > 1, the number of events is increasing over time (nonhomogenous Poisson process) beta = 1, a stable process (the mean number of events is constant over time; homogenous Poisson process) beta < 1, the number of events is decreasing over time (nonhomogenous Poisson process) The interpretation of the line should not be viewed as a precise measurement; the interpretation of the slope should be viewed within the context of having confidence limits. So a slope slightly less than or slightly more than one does not necessarily mean the number of events being tracked is increasing or decreasing significantly (due to process noise), while a slope significantly deviating from one indicates a definite trend. The interpretation of the CA plot is visually intuitive. A stable process will tend to plot as a straight line with beta approximately equal to one. If there is a change in the process, a cusp might be seen where the plotted points will begin to form a line with a different slope. When such significant shifts occur, it is advisable to plot multiple lines on the plot, instead of just one. Following is a discussion about NASA mishaps with examples of trending them over time. Mishaps are events that result in injury to personnel or damage to facilities or equipment. NASA classifies mishaps as being of A, B, C or D severity, from worst to least damaging or injurious, respectively. Mishaps are unwanted events that occur apparently at random, whose frequency may vary depending on the type of mishap. The two most prevalent types of industrial safety mishaps at all NASA Centers, in frequency of occurrence are: 1. slips, trips and falls and 2. ergonomic injuries. CA plots will be examined for both types. Example 1, Slips, Trips and Falls: Slips, trips and falls (STFs) are the most common injuries. Figure 1 is a CA plot of Type A and B (worst severity) STFs for the time period 1 June 2007 through 31 December 2010. Each triangle signifies a monthly data entry where one or more mishaps occurred. There were no Type A or B STFs in the months of November and December 2010. They have been plotted as suspensions (time periods where no event occurred) at the end of the analyzed time interval in order to account for the mishap-free period at the end of the data set. Failure to include the suspensions would result in a higher, inaccurate indication of the mishap occurrence rate. Figure 1 — Crow-AMSAA Plot of Type A and B Slips, Trips and Falls SuperSMITH® software was used to prepare this plot. The CA plot shows cumulative monthly mishaps (the horizontal axis) plotted in a log-log plot. A homogenous Poisson process should plot as an approximately straight line in log-log space. A line has been plotted based on a best fit technique recommended by the International Electrotechnical Commission (ref. 9). Lambda is the vertical intercept at time t = 1. Beta is the slope of the fitted line. The Fit-p% >= 10 indicates that this is a good fit, per the SuperSMITH® software used to generate the plot. Since we have beta = 0.96, which is close to one, for the time period shown, Agency-level slips, trips and falls of mishap Type A and B are exhibiting a slightly decreasing Poisson process. The rate of occurrence is nearly stable but trending towards improvement. The stable Poisson process, where beta = 1, is a homogenous process; is it is stationary. In industry, a stable safety mishap process (for a given type of mishap, or all mishaps) would be expected for a factory with a stable number of employees, the same level of production, and a neither improving nor declining safety record. Instability of the process, resulting in beta unequal to one, may occur due to a change in the number of employees, a change in work processes or environment, or improving or declining safety performance. Although SuperSMITH® software has built-in prediction capability for use with CA plots, the STF dataset will be used to illustrate using the Poisson distribution directly to predict the expected number of slip, trip and fall mishaps Agency-wide for 2011. The author has verified that the same answer is obtained using either method. Figure 2 shows the output from SAS JMP software for the Chi-Squared goodness-of-fit test for the STF data set analyzed in the previous CA plot. The null hypothesis that the data is from a Poisson distribution is not rejected, which provides one indication that we have an acceptable fit. The graph on the left side of the table shows the actual data in green, with a fitted theoretical Poisson distribution drawn with a red line. This graph provides additional verification that we have a reasonably good fit of the data to a Poisson distribution. The number of months, N, is equal to 43. Knowing the fitted Poisson distribution is a good model of the stochastic process generating Type A and B slip, trip and fall injuries at all the NASA Agency centers, we can predict the expected number of STF mishaps for the next year (2011) and the expected monthly distribution of them. Figure 2 — Goodness-of-Fit Test for Slip, Trip and Fall Dataset From the fitted distribution, we have lambda = 0.5349 (the average number of mishaps per month). Using this value of lambda, we can calculate the probability of the expected number of months with x mishaps using equation 1: